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1.  Introduction 


This  report  briefly  describes  the  hardware  and  software  integration  of  a  novel  optical 
augmentation  (OA)  sensor  based  on  a  commercial-off-the-shelf  charged-coupled-device  (CCD), 
a  three  diode  laser  (one  ultraviolet  [UV]/two  near  infrared  [NIR]),  and  a  pan/tilt  device  along 
with  signal  processing  software  to  realize  a  real-time  OA  sensor  for  counter-intelligence, 
surveillance,  reconnaissance  (ISR)  applications.  This  system  is  called  the  Mobile  Optical 
Detection  System  (MODS). 

For  more  information  on  the  laser  implementation,  scientific  background,  and  field  test  results, 
see  reference  1.  The  emphasis  of  this  report  is  to  provide  a  manual  for  the  end  user. 


2.  Capabilities  and  Specifications 


The  MODS  is  designed  to  be  a  lightweight  and  mobile  system  with  1-km  range.  It  can  support 
multiple  missions  including  improvised  explosive  device  (IED)  event  detection,  force 
protection,  and  insurgent  ISR  detection.  The  laser  is  not  visible  to  the  human  eye;  therefore, 
position  is  not  compromised.  It  provides  the  location  of  the  optical  return  in  terms  of  azimuth 
and  elevation.  It  can  be  set  for  continuous  surveillance  over  a  large  area,  halting  the  system  and 
sounding  a  bell  when  acquiring  a  target,  which  will  allow  the  Soldier  to  focus  on  additional 
tasks. 


3.  Technical  Objectives 


The  MODS  integrates  a  digital  camera,  three  lasers  of  varying  wavelengths,  a  pan/tilt  device, 
and  image  processing  software.  The  digital  camera  and  three  lasers  of  varying  wavelengths 
were  assembled  in  a  box  and  mounted  on  a  pan/tilt  device.  The  control  software  allows  for 
automation  of  the  surveillance  system,  the  flexibility  of  accommodating  various  observation 
and  target  scenarios,  as  well  as  the  ability  to  record  video  data. 


4.  Hardware  Integration 


The  MODS  sensor  head  was  mounted  onto  a  heavy-duty  pan/tilt  device  to  facilitate  operation  in 
an  Army-relevant  environment.  The  sensor  head  was  integrated  onto  a  pan/tilt  device  with  a 
payload  capacity  greater  than  220  kg,  355°  pan  range  and  45°  tilt  range,  as  shown  in  figure  1. 


1 


Figure  1 .  (a)  Rear  view  of  MODS  showing  electrical  transits  for  laser  power  supply,  laser  transistor- 

transistor  logic  (TTL)  modulation,  and  CCD  power  supply,  (b)  Front  view  of  MODS  showing 
window  which  facilitates  target  illumination  and  detection  return. 


5.  Software  Integration 


The  system  control  software  and  the  graphical  user  interface  (GUI)  were  developed  in  C#  using 
Microsoft  Visual  Studio  in  a  MS  Windows  environment.  The  control  software  integrates  the 
operation  and  manages  the  control  sequence  of  the  CCD  receiver,  the  three  laser  diodes,  a 
pan/tilt  device,  and  the  image  processing  algorithms  to  achieve  the  automated  MODS. 
Automation  is  achieved  by  configuring  the  system  with  the  aid  of  the  GUI  to  survey  a  selected 
area  using  the  pan/tilt  device.  The  software  consists  of  three  parts:  (1)  the  control  sequence 
software  and  image  processing  algorithms,  (2)  the  user  interface,  and  (3)  the  hardware  drivers. 
There  are  three  separate  hardware  drivers  that  (1)  control  the  pan/tilt  device,  (2)  provide  TTL 
modulation  to  the  laser  diodes  via  a  parallel  port,  and  (3)  digitize  images,  used  for  target 
detection,  from  the  CCD  receiver. 

The  user  interface  allows  the  operator  the  flexibility  to  set  up  the  MODS  for  surveillance  in 
multiple  configurations.  Moreover,  the  interface  allows  the  operator  to  choose  which 
combination  of  lasers  to  use  for  target  detection  and  modify  various  parameters  within  the 
image  processing  algorithms  to  allow  for  enhanced  target  detection.  These  and  other  features 
of  the  system  are  described  in  more  detail  in  section  6. 


6.  MODS  Control  Software  and  Image  Processing 


The  MODS  software  acquires  an  image  from  the  digital  camera.  This  is  the  background  image 
used  in  the  image  processing  algorithm.  The  first  laser  is  turned  on  and  a  second  image  (laser  1 
image)  is  acquired  from  the  camera.  The  software  then  turns  off  the  first  laser,  turns  on  the 
second  laser,  and  acquires  a  third  image  (laser  2  image)  from  the  camera.  This  process  is 
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repeated  for  the  third  and  final  laser  acquiring  the  laser  3  image.  The  background  image  is  then 
subtracted  from  each  laser  image  acquired  providing  three  sets  of  target  information.  In  order 
to  reduce  false  alarms,  these  three  sets  of  target  information  are  combined  to  generate  one  final 
set  of  target  information  to  be  displayed  and  recorded.  The  subtraction  algorithm  being  used 
allows  for  adjustment  to  the  size  and  shape  of  the  target  as  well  as  clutter  and  contrast 
thresholds. 

The  MODS  can  be  configured  to  survey  a  selected  area  using  the  pan  and  tilt  device.  The 
minimum  and  maximum  angle  in  increments  of  0.5°  for  both;  the  pan  and  the  tilt  can  be  set  to 
allow  the  system  to  monitor  a  larger  area.  The  system  automatically  cycles  through  each 
position  performing  the  previously  described  image  processing  procedure  at  each  location 
within  the  surveillance  region.  Upon  identification  of  a  target,  the  motion  of  the  pan/tilt  device 
is  halted,  a  bell  is  sounded  and  the  system  continues  to  display  the  video,  the  targets,  and  target 
locations  to  the  operator  and/or  saved  to  a  file.  This  designated  surveillance  region  can  be 
scanned  continuously  or  a  single  time.  The  pan/tilt  device  can  also  be  set  to  a  constant  position 
for  continuous  surveillance  at  that  location. 


7.  User  Interface 


The  MODS  software  initializes  in  a  mode  convenient  for  system  configuration.  When  initially 
starting  the  system,  raw  images  from  the  digital  camera  are  displayed  in  the  First  Quadrant 
allowing  the  operator  to  adjust  the  gain,  exposure,  and  focus  of  the  camera.  The  operator  can 
also  use  the  pan/tilt  controls  to  view  an  area  of  interest. 

The  GUI  interface  (figure  2)  was  designed  to  have  the  flexibility  to  allow  the  operator  to  set  up 
the  MODS  for  surveillance  in  a  variety  of  configurations.  These  configurations  include 
positioning  the  MODS  in  a  designated  direction  to  survey  and  area  without  moving  the  pan  and 
tilt;  presetting  a  larger  area  for  observation  while  using  the  pan  and  tilt  device  in  an  automated 
configuration;  and,  within  these  two  scenarios,  allowing  the  ability  to  select  which  combination 
of  lasers  to  be  used  for  detection.  The  user  interface  also  allows  the  operator  to  modify  five 
parameters  of  the  image  processing  software  making  the  system  flexible  to  target  shape,  size, 
and  reflectivity. 

The  GUI  is  broken  into  four  main  sections  and  an  Exit  button.  The  upper  left  (first)  quadrant 
displays  the  background  or  raw  image  from  the  camera,  the  upper  right  (second)  quadrant 
displays  one  of  three  processed  images,  the  lower  left  (third)  quadrant  contains  all  the  controls 
for  the  system,  and  the  lower  right  (forth)  quadrant  displays  output  information  from  the 
system. 
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Figure  2.  MODS  user  interface. 

7.1  First  Quadrant 

This  quadrant  displays  the  background  or  raw  image  acquired  from  the  digital  camera.  When 
targets  from  the  combined  laser  images  are  detected,  they  are  displayed  in  this  window  with  a 
rectangle  around  the  target  along  with  the  relative  azimuth  and  elevation  information  associated 
with  the  target.  The  azimuth  and  elevation  of  these  targets  is  also  displayed  in  the  output  panel 
in  the  fourth  quadrant. 

The  first  quadrant  can  also  be  used  to  mark  the  bore  sight  crosshair  or  the  actual  location  at 
which  the  laser  is  firing.  This  is  accomplished  with  a  simple  click  of  the  mouse  button.  The 
bore  sight  crosshair  can  be  set  or  moved  by  placing  the  mouse  pointer  on  the  desired  location  in 
the  Raw  Image  Panel  and  clicking  the  left  mouse  button.  This  x-y  parameter  is  saved  into  a 
startup  file  that  is  automatically  reloaded  when  restarting  the  MODS  system.  This  capability 
saves  the  operator  from  having  to  recalibrate  the  bore  sighting  each  time  the  system  is  restarted. 

7.2  Second  Quadrant 

This  quadrant  displays  one  of  the  three  processed  images.  The  displayed  image  is  the  result  of 
the  subtraction  algorithm  on  the  background  image  and  the  selected  laser  image.  If  the  image 
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processing  performed  on  the  selected  laser  and  background  images  produce  targets,  they  are 
displayed  with  crosshairs  and  the  relative  azimuth  and  elevation  data  in  this  window.  The 
crosshairs  and  relative  location  data  for  laser  1  are  violet,  for  laser  2  are  red,  and  laser  3  are 
Indian  red  (light  red).  Associating  each  laser  with  a  unique  color  assists  the  operator  to  rapidly 
discern  which  processed  laser  image  is  being  viewed. 


7.3  Third  Quadrant 

The  third  quadrant  (figure  3)  contains  the  controls  for  the  entire  MODS.  It  is  the  most 
complicated  quadrant  and  is  broken  into  five  smaller  control  panels  including  Camera, 
Platform,  Pan  Tilt  Controls,  System,  and  Detection.  The  buttons,  check  boxes,  and  drop-down 
menus  in  this  quadrant  are  disabled  if  they  are  not  to  be  accessed  while  the  various  processes 
are  running. 


Camera 

Gain  Normal  v 


Exposure: 


Platform 

Pan  Mm  -30  C  TiMm  -2  C 

Pan  Max  30  Tilt  Max  2 

Pan  Step:  3  Tilt  Step:  2 

Pan  o  Tit  0 

System 

0  Continuous  Scan  Processed  View  Laser  1  v 
!  I  Mute  Detection  Bel  0  Normalize  Image 


Grab  Stop 


Laser  Marker  ON 


15  ms 


Pan  Tilt  Controls 


Tilt  Up 


Home 


Pan  Left 


STOP 


Pan  Right 


Tilt  Down 


Set  Center  Pont 


Detection 

Minirmm  Detection  Contrast:  40 

Minimum  Detection  Size:  20 

Maxmum  Detection  Size:  200 

Detection  XV  Pixel  Ratio:  2 


Start  Recotdng 

□utter  Filler:  3 

A 

V 

0  Laser  1 

0  Laser  2 

0  Laser  3 

Azimuth 

Elevation: 

Figure  3.  Third  quadrant. 

7.3.1  Camera 

The  camera  control  panel  allows  the  operator  to  send  commands  directly  to  the  digital  camera. 
The  operator  can  change  the  gain  of  the  camera  from  normal  to  high  using  the  drop-down  menu 
box.  The  slider  bar,  in  combination  with  the  text  box,  allows  the  operator  to  adjust  the 
exposure  time  (microseconds)  of  the  camera.  The  operator  can  choose  to  move  the  slider  bar  or 
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enter  the  exposure  time  directly  into  the  text  box.  Modification  both  the  gain  and  the  exposure 
is  allowed  while  processing  and  displaying  images. 

7.3.2  Platform 

The  platform  panel  allows  the  operator  to  set  up  the  MODS  for  surveillance  of  a  larger  area. 

The  three  Pan  drop-down  menu  boxes  allow  the  operator  to  set  a  minimum  angle,  maximum 
angle,  and  a  step  increase  angle  for  panning  the  pan/tilt  device.  The  three  Tilt  drop-down  menu 
boxes  allow  the  operator  to  set  a  minimum  angle,  maximum  angle,  and  a  step  increase  angle  for 
tilting  the  pan/tilt  device.  The  Pan  and  Tilt  text  boxes  continually  update  displaying  the  current 
angle  of  azimuth  and  elevation  of  the  pan/tilt  device. 

7.3.3  Pan/Tilt  Controls 

The  pan/tilt  control  panel  contains  all  the  buttons  to  control  the  pan/tilt  device.  Each  press  of 
the  Tilt  Up  button  or  Tilt  Down  button  allows  the  operator  to  tilt  the  pan/tilt  device  one 
increment  in  the  desired  direction.  The  angle  size  of  this  increment  is  specified  by  the  Tilt  Step 
size  in  the  Platform  panel.  Each  press  of  the  Pan  Left  button  or  Pan  Right  button  allows  the 
operator  to  pan  the  pan/tilt  device  one  increment  in  the  desired  direction.  The  angle  size  of  this 
increment  is  specified  by  the  Pan  Step  size  in  the  Platform  panel.  The  STOP  button  allows  the 
operator  to  send  a  stop  command  to  the  pan/tilt  device  at  any  time.  The  STOP  button  is  always 
enabled.  The  Set  Center  Point  button  allows  the  operator  to  set  a  (0,  0)  or  home  point  for 
azimuth  and  elevation.  This  center  point  is  then  used  by  the  Home  button  to  return  the  pan/tilt 
device  back  to  its  (0,  0)  position.  The  Process  Images  button  allows  the  operator  to  acquire 
images,  fire  the  lasers,  and  process  the  images  for  target  detections  without  moving  the  pan/tilt 
device  once  it  has  been  pointed  in  the  desired  direction.  Once  this  process  is  started  the  Process 
Images  button  becomes  the  Stop  button,  which  ends  this  process  when  clicked. 

7.3.4  System 

The  System  panel  allows  the  operator  to  manipulate  many  aspects  of  the  MODS.  The 
Continuous  Scan  check  box  allows  the  operator  to  choose  to  scan  the  area  selected  in  the 
Platform  panel  repeatedly  when  checked  or  one  single  scan  from  min  to  max  of  pan  and  tilt  if 
not  checked.  When  checked,  the  Mute  Detection  Bell  check  box  silences  the  bell  sounded 
when  a  target  is  detected.  The  Processed  View  drop-down  menu  box  allows  the  operator  to 
select  which  laser  image  to  be  displayed  in  the  Processed  Image  panel  of  the  second  quadrant. 
This  displayed  image  is  actually  the  result  of  the  image  detection  algorithm  (subtraction 
algorithm)  on  the  background  image  and  the  selected  laser  image.  The  Normalize  Image  check 
box,  if  checked,  allows  the  operator  to  view  the  image  being  displayed  in  the  Raw  Image  Panel 
of  the  First  Quadrant  after  normalizing  the  image.  The  normalization  is  for  display  only  and 
does  not  affect  the  image  data  being  used  by  the  target  detection  algorithm. 

The  Grab  Images  button  allows  the  operator  to  view  images  acquired  by  the  camera  without 
firing  the  lasers  or  performing  the  image  processing.  This  feature  allows  the  operator  to  view 


6 


images  while  focusing  the  camera  or  planning  the  area  of  surveillance.  The  Grab  Images  button 
becomes  the  Grab  Stop  images  button,  which  will  stop  the  camera  from  acquiring  images  when 
clicked.  The  MODS  starts  up  with  the  Grab  Images  button  selected.  The  operator  must  click 
the  Grab  Stop  button  to  end  the  process  of  acquiring  and  displaying  images  before  beginning 
either  of  the  target  acquisition  processes  by  clicking  the  Process  Image  button  in  the  Pan  Tilt 
Controls  panel  or  the  Start  button  in  the  System  panel. 

The  Laser  Marker  ON/OFF  button  allows  the  operator  to  view  or  not  view  the  bore  sighting 
crosshair  in  the  Raw  Image  panel  of  the  First  Quadrant. 

The  Start  button  in  this  panel  starts  the  complete  MODS  surveillance  system,  moving  the 
pan/tilt  device  through  the  various  locations  designated  by  the  information  input  into  the 
platform  panel.  The  pan/tilt  device  continues  to  move  through  its  various  settings  until  it 
detects  a  target.  At  this  time,  the  device  stops  panning  and  tilting,  and  continues  the  processes 
of  firing  the  lasers  and  processing  the  images  pointing  only  in  that  direction  until  the  Resume 
button  is  clicked.  When  the  Resume  button  is  clicked,  the  pan/tilt  device  continues  stepping 
through  its  various  locations  until  it  detects  another  target.  The  Start  button  becomes  the  Stop 
button,  which  ends  the  above  process  when  clicked. 

The  Start  Recording  button  saves  the  images,  as  well  as  any  bore  sighting  crosshairs,  target 
rectangles,  and  target  azimuth  and  elevation  data,  being  displayed  in  the  Raw  Image  Panel  of 
the  First  Quadrant  to  a  file  selected  using  a  pop-up  file  select  window  (figure  4).  The  Start 
Recording  button  becomes  the  Stop  Recording  button,  which  is  clicked  when  the  operator  no 
longer  wants  to  record  the  data. 
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Figure  4.  File  select  pop-up. 

The  Laser  check  boxes  denote  which  lasers  will  be  included  in  the  system  configuration.  Laser 
1,  Laser  2,  and  Laser  3  are  fired  and  used  in  the  final  combination  for  a  true  detection  only 
when  its  corresponding  check  box  is  checked.  A  true  detection  is  defined  as  one  in  which  a 
retro-reflection  is  received  from  all  the  lasers  selected. 

7.3.5  Detection 

The  Detection  panel  allows  the  operator  to  modify  five  variables  used  by  the  image  processing 
and  target  detection  algorithm.  Using  a  drop-down  menu  box  the  operator  can  change  the  value 
for  minimum  detection  contrast,  minimum  target  detection  size  (pixels),  maximum  target 
detection  size  (pixels),  and  the  x-y  pixel  ratio  of  a  desired  target,  as  well  as  adjust  the 
parameters  of  a  clutter  (low  pass)  filter.  These  variables  help  to  eliminate  false  alarms. 

The  Minimum  Detection  Contrast  variable  eliminates  targets  whose  reflection  does  not  reach  a 
minimum  threshold  for  brightness. 

The  Minimum  Detection  Size  variable  eliminates  reflections  that  are  too  small  to  be  considered 
valid  targets.  The  Maximum  Detection  Size  variable  eliminates  reflections  that  are  too  large  to 
be  considered  valid  targets.  The  ability  to  modify  the  valid  detection  size  variables  allows  the 
system  more  flexibility  in  accommodating  various  ranges  from  the  area  of  surveillance  and 
target  scenarios. 
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The  Detection  X-Y  Pixel  Ratio  variable  to  eliminate  reflections  based  on  shape.  For  example, 
if  the  desired  target  is  circular,  then  setting  the  X-Y  Pixel  Ratio  to  1.5  or  2  will  eliminate  false 
alarms  that  are  long  and  thin  and  could  be  caused  by  edges. 

The  Clutter  Filter  variable  is  used  by  the  low  pass  filter  of  the  image  processing  algorithm.  It 
ranges  from  0  to  100.  A  larger  value  results  in  stronger  filtering.  A  value  of  zero  turns  the 
filtering  off. 

The  text  boxes  in  this  panel  display  the  azimuth  and  the  elevation  of  the  location  of  the  most 
recent  target  detected. 

7.4  Fourth  Quadrant 

This  quadrant  displays  output  information  from  the  MODS  such  as  errors,  camera  control 
feedback,  and  pan/tilt  device  feedback.  Most  importantly  to  the  operator,  it  displays  azimuth 
and  elevation  of  the  detected  targets  which  are  displayed  in  the  first  quadrant. 


8.  Exit  Button 


The  exit  button  shuts  down  the  system  when  clicked,  if  all  processes  have  been  stopped.  If  the 
exit  button  is  clicked  while  any  of  the  processes  are  in  progress,  it  has  a  safety  feature  of  a  pop¬ 
up  box  (figure  5)  notifying  the  operator  that  processes  are  still  running.  The  pop-up  box  allows 
the  operator  to  click  the  Cancel  button,  which  allows  the  operator  to  continue  working  on  the 
MODS  or  click  the  OK  button,  which  properly  ends  all  processes  in  progress  before  shutting 
down  the  MODS. 


Figure  5.  Exit  pop-up. 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


CCD 

charged-coupled-device 

GUI 

graphical  user  interface 

IED 

improvised  explosive  device 

ISR 

intelligence,  surveillance,  reconnaissance 

MODS 

Mobile  Optical  Detection  System 

NIR 

near  infrared 

OA 

optical  augmentation 

TTL 

transistor-transistor  logic 

UV 

ultraviolet 
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